On optimization, parallelization and convergence of the Expectation-Maximization algorithm for finite mixtures of Bernoulli distributions

This paper reviews the Maximum Likelihood estimation problem and its solution via the Expectation-Maximization algorithm. Emphasis is made on the description of finite mixtures of multi-variate Bernoulli distributions for modeling 0-1 data. General ideas about convergence and non-identifiability are presented. We discuss improvements to the algorithm and describe thoroughly what we believe are novel ideas in the treatment of the topic: 1) identification of unique data points and recycling of that information 2) parallelization of the algorithm in a multi-threaded fashion 3) cluster assignment options. Experiments demonstrate that most of our approaches produce good results and encourage further research on the topic.